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A Post-Class Question Strategy That Provides Feedback 
and Connects In- and Out-of-Class Learning 


Alison B. Flynn 
University of Ottawa 


An instructional method is described that was used in a large, introductory organic chemistry course 
to ask online, post-class questions and to use the students’ answers to design the lesson and learning 
activities for the following class. Additional goals of this method were to provide relevant, regular, 
and promptfeedback to students and the instructor, to emphasize the value of time on task (i.e., that 
learning takes time) and to connect in- and out-of-class learning. Herein, the results of formative 
project evaluation and preliminary results of an educational evaluation of the method are also 
described. 


Introduction 

M any strategies have been developed to actively 
engage students in large classes (Boehmler 
& Smith, 2006; Caldwell, 2007; Cameron, 1999; 
Clouston & Kleinman, 1999; Cutts, Kennedy, 
Mitchell, & Draper, 2004; Harley, Maher, Henke, 
& Lawrence, 2003; Harpp, 1994; Lyon & Lagowski, 
2008; Rutherford, 2004); however, it remains 
challenging to provide students with relevant, 
individualized feedback, to emphasize the value of 
time on task (Chickering & Gamson, 1987), and to 
connect in- and out-of-class learning. 

Recently, online organic chemistry 
homework programs (ACE Organic: Achieving 
Chemical Excellence; Connect: Teaching and 


Learning Program; OWL: Online Web Learning; 
Reaction Explorer: Organic Chemistry Tutorials; 
WileyPlus: Courses in Organic Chemistry) have been 
developed that provide some important advantages 
over other homework software. The programs have 
questions that ask students to draw their own answers 
(molecules), which provides a valuable addition to 
multiple choice, matching, ranking, and numeric 
questions. The software is capable of molecular 
recognition (i.e., of recognizing the molecule that the 
student has drawn and comparing it to the correct 
answer), which allows the program to give tailored 
feedback that is based on the students own answer. 
Using the gradebook, the instructor can review the 
answers provided by each student, thereby obtaining 
information with respect to the students’ level of 
understanding. 
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Even though I assigned weekly quizzes from 
one of the aforementioned homework programs 
(ACE Organic: Achieving Chemical Excellence) and 
reviewed the more difficult questions in class, I felt 
that the quizzes were somewhat disconnected from 
the in-class work. This disconnection was especially 
evident when the difficult questions were reviewed 
long (up to two weeks) after the topic had been 
covered in class. Furthermore, and not surprisingly, 
discussion forum statistics (where students asked 
many quiz-related questions) peaked in the hours 
before the quiz was due. This surge in participation 
suggested that students were not reviewing the 
concepts discussed in-class immediately following the 
class, which would be ideal to cement their learning 
(Chickering & Gamson, 1987). 

To address these concerns, I developed 
a post-class question method with the intent of 
satisfying the following objectives: 1) to encourage 
students to spend time on task; 2) to connect in- and 
out-of-class learning; 3) to provide prompt, regular, 
and relevant feedback to the students; and 4) to 
provide feedback to the instructor. This method is 
based on scaffolding learning theory, which involves 
supporting students through the learning process by 
appropriately sequencing content, tasks, and teacher 
and peer support until students can apply new skills 
and strategies independently (Larkin, 2002). Herein, 
I have described the method and the initial results of 
the evaluation of the method’s effectiveness. 

Background 

The courses described herein, Organic Chemistry 
I and Organic Chemistry //, are large (250-420 
students per section), first- and second-year organic 
chemistry courses at the University of Ottawa. They 
are generally considered to be challenging courses. 
Approximately 67% of students enrolled are from the 
Faculty of Science, 21% from the Faculty of Health 
Sciences, 6% from the Faculty of Engineering, 5% 
from the Faculty of Social Sciences, and less than 
1% from other faculties such as the Faculty of Arts. 
Organic Chemistry I and II are required courses for 
the majority of students’ enrolled and are required 


for admission to many graduate programs and 
professional schools, such as medicine. 

Each course involved weekly quizzes and twice 
weekly post-class questions using ACE Organic, an 
online homework program (ACE Organic: Achieving 
Chemical Excellence), clickers (Bhattacharyya, 
2006) (except in 2008 when clickers were not used), 
two midterms and a final exam. The first-year course 
also had a three-hour laboratory component. In each 
course there were two 80-minute lectures per week, 
optional 80-minute tutorial sessions, and optional 
written assignments for which answers were posted 
on the course website. 

Description of the Method 

I assigned an online post-class question after each 
class; an example is shown in Figure 1. The question 
probed the students’ understanding of a subject and/ 
or concept taught in that class. The students had two 
attempts to answer the question and the program 
provided feedback based on their response. In the first 
two courses (while the method was being developed) 
the post-class questions were worth bonus marks only 
(up to 1% of the student’s final mark). In 2010 and 
2011, post-class questions were worth 2% of the final 
grade. The student was accorded 100% for a correct 
answer, 90% for an incorrect answer, and 0% if he/ 
she did not attempt the question. 

I reviewed the students’ answers prior to the 
following class, and used these answers to design the 
first segment of the next class. If most of the students 
had obtained the correct answer, then we spent very 
little class time, if any, discussing the topic related 
to the post-class questions. However, if students 
struggled to answer the question, then I used the 
students’ answers to design a clicker question, 
typically in multiple-choice format, which contained 
the correct answer along with the most common 
incorrect ones. If students struggled to answer the 
clicker question, then I asked them to pair up and 
to discuss the problem. I asked the clicker question a 
second time and the results of that question drove an 
in-class discussion, which concluded that section. An 
overview of the method is shown in Figure 2. 
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Post-class question from March 10 

Maximum allowed tries per question: 2 

(1) (Question #1326] An alcohol is converted to Its tosylate (a sulfonate ester), then allowed to 
react with a strong, UNhindered base to give the alkene shown. Draw the starting alcohol, 


Click Image to launch MarvinView 1 * 



Your response Is Incorrect {1 try}. 

What would Zaitsev say about the direction in which this alcohol would eliminate? 

[ Question List j [ Assignments List] 



[ Submit Response") 


Figure i 

A post-class question with feedback for an incorrect response. Source: ACE Organic. Used with permission. 
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Figure 2 

Online post-class question method 
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I typically spent 5-10 minutes creating each 
question, 2-10 minutes reviewing students’ answers, 
and 5-10 minutes to creating a new clicker question 
(which I did for approximately two thirds of the 
classes). At the end of the semester, I spent about 
20 minutes calculating the grades by exporting the 
data from the ACE gradebook to an excel spreadsheet 
then assigning the marks as described above. 

I started assigning post-class questions a few 
weeks into each class (to give students time to get used 
to the course and its requirements) and, subsequently, 
at least one post-class question was assigned after every 
class. Students could answer a question correctly in 
as little as two minutes, depending on their levels of 
proficiency and commitment to reviewing/learning the 
lesson in question. Unfortunately, ACE organic does 
not provide statistics on the amount of time taken for 
a student to complete a question/assignment. Students 
could also review and/or re-do the questions as part of 
the preparation for midterms and final exams. 

Project Evaluation 

I designed an evaluation of this project, which is 
currently underway, largely based on the format 
suggested in the National Science Foundation’s 
2010 User-Friendly Handbook for Project 
Evaluation (Frechtling et al., 2010). The first phase 
of the evaluation involved a formative evaluation 
of the method; the second phase is a summative 
evaluation. The results of the formative evaluation 
phase are described below, which sought to 


answer the following questions: 1) What was the 
participation rate in each class? 2) What were the 
students’ opinions and suggestions with respect to 
the aforementioned project objectives? 3) What 
were the instructor’s opinions and suggestions with 
respect to the aforementioned project objectives? The 
summative evaluation phase is currently underway, 
in which the key research questions are: 1) What is 
the degree of improvement in learning outcomes, if 
any? 2) What is the most effective way of using the 
post-class question method? The preliminary results 
of the summative evaluation are described below. 

Participation rates 

The participation rates varied from 61.4% to 75.1% 
(Table 1) and were higher in the second-year course 
than in the first-year course. In 2010, I increased 
the weighting of the post-class questions to 2% of 
the students’ final grade; they had previously been 
worth bonus marks only. There was a concurrent 
improvement in participation rates on post-class 
questions when comparing rates between different 
cohorts of the same course (Table 1). In every course, 
at least 25% of the class did not participate. While 
the reasons for this have not yet been explicitly 
probed, it is perhaps because of the low weighting 
of the post-class questions (bonus marks or 2% of 
the students’ final grade). Class attendance did not 
change significantly with the addition of post-class 
questions, which could be attributed to the fact that 
clickers were already in use in the class (for which 
participation marks were assigned), the fact that 
lecture notes were not available online for the course 


Table i 

Participation rates on post-class questions 


Course 

Number of post¬ 
class questions 

Average participation rate on 
post-class questions (%) 

Standard 
deviation (%) 

Organic Chemistry 1 2009 

10 

61,4 a 

9.1 

Organic Chemistry 1 2011 

15 

68.5 a 

9.8 

Organic Chemistry II 2008 

15 

65.8 b 

9.4 

Organic Chemistry II 2010 

18 

75.1 fc 

12.5 


£(20) = 1.85, p = 0.0394. b £(31) = 2.44, p = 0.0104. 
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(perhaps providing motivation for students to come 
to class), and/or other reasons. 

The feedback received on student surveys 
has been extremely positive. A selection of students’ 
responses is shown in Figure 3, below. Specifically, 
students responded that the post-class questions 
helped them learn class material, that they reviewed 
their notes regularly in order to answer the post-class 
questions, and that they gave a reasonable effort to 
respond to the questions. 

The students’ comments were also very 
positive. For example students said that: 

“Post-class questions really forced me 
to review the stuff i [sic] learned in 
class, so i think it was a great idea.” 

“Also going over the [questions] 
which most people had trouble with 
in class was also good, because then 


we could see exactly where we went 
wrong.” 

“The post-class questions were great 
to encourage at least a little 'daily’ 
reviewing, and again, I think it’s 
great that we were rewarded just for 
trying.” 

Very few technical difficulties were reported, the 
main one being that the program was “picky” and 
so a slight error in the student’s answer, which might 
be unrelated to the student’s understanding of the 
concept being tested, was nevertheless recorded as 
incorrect. Most students learned to draw organic 
molecules on a computer for the first time in Organic 
Chemistry I and they could do so through online 
tutorials before attempting an assignment. Fewer 
than five students per course reported technical 
difficulties related to drawing/submitting answers. 


50% 



ACE post-class I reviewed my I gave a 
questions class notes reasonable 
helped me learn regularly in effort to answer 
course material order to answer the ACE post- 
the ACE post- class questions 
class questions 

Survey statement 


Strongly agree 

■ Agree 

Neither agree nor disagree 

■ Disagree 

Strongly disagree 


Figure 3 

Selected student survey results from Organic Chemistry I, 2009 (N-604) 
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Instructor's opinions and suggestions 

Overall, I feel that the time required to create the 
questions, review students’ answers, and design 
follow-up questions/discussion topics was well 
worth the effort. It was helpful to see students’ 
answers regularly instead of just after midterms/ 
exams and to be able to address common errors 
so easily and often. It also seemed that student 
engagement had improved in the class, which was 
particularly observed with more students asking 
questions about the last class’ lesson and having 
more students contributing to in-class discussions 
(anecdotal, qualitative results). 

The largest technical difficulty on my part 
arose with the release time of the questions (i.e., when 
the question became available to students). With the 
program that we were using, I could not specify a 
specific time to release the question, and so there was 
occasionally a delay between the end of the class and 
the posting of the question, depending on how long 
it took me to get back to a computer. This was a very 
minor problem and for the most part the post-class 
questions were straightforward to assign and review. 


Comparison of learning outcomes 

Preliminary results at the question level suggested 
that learning outcomes have improved. For example, 
Table 2 shows a post-class question (for which answer 
choices were not provided) in which I asked students 
to draw the first organic intermediate for the reaction 
shown. Initially, only 27% of students obtained the 
correct answer. When I asked the same question at the 
beginning of the following class as a clicker question, 
with the correct answer along with the most common 
incorrect answers shown, 55% of students obtained 
the correct answer (*(766) = 8.44l,/> < 0.0001). After 
peer discussion, 88% of students gave the correct 
answer to the same question (*(652) = 10.826, p < 
0.0001). Although the same question was asked each 
time, Mazur has shown that students often arrive at the 
correct answer after peer discussion, especially when 
30 - 70% of students have initially answered correctly, 
and even if the correct answer was not originally the 
predominant answer (Mazur, 1997). 

A comparison of the final exam results 
between 2008 and 2009 for question types (S N 1) that 
were similar to the one shown above (Table 2) also 


Table 2 

Distribution of results in response to the question: "Draw the first organic 
intermediate formed in the reaction shown below." Organic Chemistry I, 2009 



Method used to ask the question 

A 

Student answers (%) 

B C 

D 

Online ACE post-class question (N=439) a 



27 


Clicker question (individual) (N=380) b 

18 

9 

55 

18 

Clicker question (after peer discussion) (N=380) b 

2 

0 

88 

10 

Final exam question (N=606) a,c 



80 


Final exam question, 2008 (N=680) cd 



67 



a Data from 2 sections. b Data from 1 section. c The question was a variation on the same topic. 
d Organic Chemistry II (Data from 2 sections). 
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suggested an improvement in learning outcomes. 
While the question shown in Table 2 was given as a 
post-class question in the Organic Chemistry I class 
in 2009, a question of this type was not asked as a 
post-class question in the Organic Chemistry II class 
in 2008. An approximately equal amount of time was 
spent in class on the topic on this topic (i.e., the S N 1 
reaction) in both courses. In 2009, 80% of students 
were able to draw the correct answer on a different 
question of the same type as the post-class question. 
This was in contrast to the 67% of students in 2008 
who were able to draw the correct answer (7(1281) = 
5.356,< 0.0001). 

A correlation between post-class questions 
and grades, but not necessarily that the post-class 
questions have caused the improvement in grades. 
There was no statistically significant difference 
in the students’ final grades or GPAs between 
cohorts that used post-class questions and those 
that did not (which was provided to the author, 
stripped of all student identifying information, by 
the Institutional Research and Planning office at 
the University of Ottawa). This was not surprising 
given the number of factors that can affect a final 
grade or GPA, particularly between different courses 
or different cohorts of students. Nevertheless, the 
promising results thus far are encouraging and more 
investigation is merited. 

Concluding Remarks 

Asking online post-class questions after every class 
encouraged students to review each class immediately 
upon its completion and the online homework 
program provided them with feedback tailored to 
their individual answers. I used this method to gauge 
the degree of students’ understanding after every 
class, identify common errors, and tailor subsequent 
classes accordingly. Preliminary project evaluation 
results suggest that this method had a positive 
impact on student learning, although there was not 
a statistically significant difference in students’ final 
grades following the introduction of the method. 
The merits of this method are currently being more 
fully evaluated. 
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